Mapping correlated Gaussian patterns in a perceptron

We study the performance of a single-layer perceptron in realising a binary mapping of Gaussian input patterns. By introducing non-trivial correlations among the patterns, we generate a family of mappings including easier ones where similar inputs are mapped into the same output, and more difficult ones where similar inputs are mapped into different classes. The difficulty of the problem is gauged by the storage capacity of the network, which is higher for the easier problems. The use of statistical mechanics techniques in the analysis of feedback neural networks has led to a deep understanding of the equilibrium properties of these systems (Amit et a1 1987). Feedback neural networks, e.g. the Hopfield-Little model (Hopfield 1982, Little 1974), have a non-trivial dynamics which possesses a huge number of attractors. Part of these attractors can be imprinted in the network through a learning procedure which specifies the strengths of the couplings between the neurons, thus allowing the network to be used as an associative memory (Hopfield 1982). On the other hand, single-layer feedforward neural networks, e.g. Rosenblatt’s perceptron (Rosenblatt 1962), have a rather dull dynamics but a very rich learning process which had been fully studied in the 1960s through rigorous mathematical analysis, simulations on digital computers and by constructing an actual machine (Block 1962, Minsky and Papert 1969). Recently Gardner (1988) and Gardner and Derrida (1988) have successfully rederived some of the results concerning the maximum storage capacity of the perceptron in the framework of the equilibrium statistical mechanics (see also Opper (1988, 1989) for a more recent contribution). The architecture of the perceptron considered in those studies is shown in figure 1. The input layer consists of N neurons {ti = *l, i = 1, . . . , N } , each one connected to the output neuron S = *1 through the couplings J i .